Assessing critical attributes of prospective teachers: Implications for selection into initial teacher education programmes

In this article, we present an overview of a research programme that is focused on reliably and validly measuring non-cognitive attributes associated with teacher effectiveness for the purpose of selection into ITE programmes. The novel contribution of this programme of research is that it builds on research and theory from educational psychology, methodology from organisational psychology, and findings from selection practices in medical education to address a critical educational problem. This article contributes four new insights into the selection of teachers and prospective teachers. First, we provide an overview of a dynamic interactionist view of the formation of effective teachers. Second, we describe how context, learning opportunities, and personal characteristics work together to influence student and teacher outcomes. Third, we introduce a conceptual model of how teacher selection measures are related empirically and conceptually to teacher effectiveness. Finally, we show how theory and research on teacher selection might be implemented in a six-stage selection process. Research into teacher selection has the potential to contribute to our understanding of the psychological factors associated with teacher effectiveness and to improve the quality of teachers entering the profession. Assessing critical attributes of prospective teachers: Implications for selection into initial teacher education programmes Teachers play a more important role in influencing student learning than almost any other factor. In Hattie’s (2009) review of the factors associated with student learning and achievement, ‘teacher factors’ made a stronger contribution to student learning than home, curriculum, student, or school factors. Although teachers are linked with improvements in student learning, they are not ‘interchangeable parts’ (Weisberg et al., 2009, p.9), and individual differences in teacher effectiveness mean that student outcomes are significantly and reliably associated with who is doing the teaching. Atteberry’s work on effectiveness within large cohorts of newly-hired teachers (Atteberry, Loeb & Wyckoff, 2015) shows that teachers’ relative effectiveness is stable; that is, effectiveness measured at the very start of a career is predictive of effectiveness later in a career, and these measures are especially predictive for those who initially display the highest and lowest levels of effectiveness. The evidence for individual differences in teacher effectiveness is persuasive (Atteberry et al., 2015; Chetty, Friedman & Rockoff, 2014; Hanushek & Rivkin, 2012; Xu, Özek & Hansen, 2015), Assessing critical attributes of prospective teachers. 5–22 BJEP Monograph Series II, 12 © 2017 The British Psychological Society Correspondence concerning this article should be addressed to Robert M. Klassen, Department of Education, University of York, Heslington, York, United Kingdom YO10 5DD (email: [email protected]) 6 Robert M. Klassen & Lisa E. Kim yet school systems are often reluctant to publicly acknowledge that individual teachers vary in their levels of effectiveness (Scott & Dinham, 2008; Weisberg et al., 2009). Identifying the cognitive and non-cognitive attributes associated with teacher effectiveness is a question that educational psychology researchers have tackled for several decades. In the UK, the Department for Education guidance for initial teacher education (ITE) programmes mandates that attention is paid to cognitive (‘appropriate intellectual and academic capabilities’) and non-cognitive (‘personal qualities, attitudes, ethics and values’) attributes (Department for Education, 2016, p.11) attributes of candidates. In other settings, including high-performing education systems such as Finland and Singapore, considerable attention is paid to measuring cognitive and non-cognitive attributes for the purpose of selection into ITE programmes (Sahlberg, 2015; Sclafani, 2015). For educational psychologists – most comfortable working with theory and learning processes (Berliner, 2006) – applying research findings to real world problems, such as the selection of candidates for ITE programmes, presents a real challenge. Educational psychologists have found non-cognitive attributes such as self-efficacy (Klassen & Tze, 2014), personality (Kim & MacCann, 2016; Robertson-Kraft & Duckworth, 2014), and teachers’ beliefs about the nature of teaching and learning (Fives & Buehl, 2008; Sosu & Gray, 2012) to be associated with teacher and student outcomes. But can attributes measured at selection into ITE reliably predict a teacher’s future effectiveness? Cognitive attributes (e.g. academic achievement, literacy and numeracy skills, subject area knowledge, and pedagogical knowledge) are widely and systematically assessed in high-stakes selection settings through, for example, examination of school records or administration of standardised tests. Non-cognitive attributes, on the other hand, are not as widely and systematically assessed and are difficult to measure given assessment problems, such as response biases and faking (Johnson & Saboe, 2011). The selection of prospective teachers benefits from multiple predictors because teacher effectiveness is multidimensional and may not be easily predicted using a single predictor (Harris & Rutledge, 2010; Hattrup, 2012). The purpose of this article is to present a research programme focused on the practical problem of the selection of prospective teachers. The research programme is built on theory and research not just from educational psychology, but also from organisational psychology and from well-developed selection practices in other professional disciplines. Selection research in other disciplines Selection research is well established in fields outside of education. Ryan and Ployhart’s (2014) review of the last 100 years of selection research noted that selection research is a mature field that continues to answer fundamental questions (What should be assessed? How should we assess it?) with three key trends. First, more research is focusing on expanding the criterion space, that is, establishing a broader definition of what ‘success’ at work or in training means and how it can be conceptualised and measured. Second, researchers are continuing to develop approaches to measure diverse non-cognitive constructs, including emotional intelligence, social skills, integrity, and personality. Third, methodological advances in selection research include a new emphasis on situational judgement tests (SJTs) as a viable methodology to predict important outcomes in a range of contexts. Ryan and Ployhart also forecast future developments in selection research to include a shift from a Western-centric to a multicultural view of the key attributes targeted in selection, and an increasing emphasis on technological innovation in selection methods. 1 We use the term non-cognitive attributes to refer to within-person variables including traits, motivation, personality, beliefs, attitudes, and dispositions. The term ‘non-cognitive’ is used in contrast to ‘cognitive’ factors such as subject area knowledge or reasoning ability that are routinely collected (academic transcripts, SAT/ GRE) and used to inform selection decisions. In some literature, the term ‘non-academic’ is used in place of the term ‘non-cognitive.’ Assessing critical attributes of prospective teachers 7 Selection research in medicine Research on selection into medical training programs benefits from a well-established research base. Longitudinal predictive validity studies show that non-cognitive attributes assessed at the point of entry into a training programme are significantly associated with academic and professional effectiveness several years after selection (e.g. Lievens & Sackett, 2012). Measures of non-cognitive attributes tend to provide incremental validity – i.e. a significant increase in prediction – over and above cognitive predictors such as achievement test scores and educational background (e.g. Patterson, Ashworth, Kerrin & O’Neill, 2013). Although the selection landscape in medicine differs in important ways from that in education (e.g. level of competition among candidates, work context, and cost of training), research on selection practices for medical training can provide useful insights when designing research on selection practices for ITE programmes. A recent review systematically evaluated the methods used to measure cognitive and non-cognitive attributes of candidates for selection into medical training (Patterson et al., 2016). Of the eight selection methods identified to assess cognitive and non-cognitive attributes (aptitude tests, academic records, personal statements, reference letters, SJTs, personality and emotional intelligence assessments, traditional interviews, multiple mini-interviews, and selection centres), four methods were deemed to be the most effective and fair methods. For measurement of cognitive attributes, examination of academic records and aptitude tests (e.g. MCAT or UKCAT) was found to be effective and fair. For measurement of non-cognitive attributes, SJTs, multiple mini-interviews, and selection centres were found to be effective (higher predictive validities) and fair (leas prone to bias). Less effective selection methods included reference letters, personal statements, and traditional interviews, with sparse research either for or against the use of personality assessments and emotional intelligence for selection. The review concluded that although considerable progress had been made in selection research during the period covered (1997–2015), a distinct lack of theory-informed research was inhibiting the development of a richer understanding of how cognitive and non-cognitive attributes contribute to competence and career success. We suggest that ITE selection research has been inhibited by a similar lack of theory-informed research. Are ‘good teachers’ born or made? The aim of ITE selection is to identify candidates with characteristics (whether fixed or malleable) that are associated with effective teaching. When ITE selectors are choosing a restricted number of candidates from among a pool of candidates, they use two evaluative processes: (a) they evaluate candidates’ background factors, cognitive attributes, and non-cognitive attributes; and (b) they evaluate candidates’ potential for developing these attributes during the ITE program and early teaching career. Some attributes, such as self-efficacy (Gutman & Schoon, 2013), may be more malleable than other attributes, such as personality (Tucker-Drob & Briley, 2014). Selection involves determining the extent to which candidates display the desired attributes (the ‘born teacher’) and/or the potential to develop the desired attributes (the ‘made teacher’). Educational researchers have labelled the belief that teachers are born and not made a ‘damaging myth’ that results in policies that rely on ‘some kind of prenatal alchemy’ (Darling-Hammond, 2006, p.ix) to identify and prepare effective teachers. Scott and Dinham (2008) labelled this belief a ‘nativist myth’ (p.115) that is widely – and, in their view, incorrectly – held by many teachers. However, many educational psychologists (e.g. Kunter, Kleickmann, Klusmann & Richter, 2013) believe that personal characteristics such as intelligence, motivation (including competence beliefs), and personality interact with environmental factors such as professional learning opportunities to shape pre-service and practicing teacher professional outcomes. 8 Robert M. Klassen & Lisa E. Kim Three views of the formation of effective teachers The born-or-made question has important implications for teacher selection. Borrowing from the language of Dweck’s implicit theories of intelligence (1986), some hold an incremental view about the factors leading to teacher effectiveness while others hold an entity view of the attributes. In Figure 1, we highlight the relevance of selection from three viewpoints: (a) the incremental view (referred to as the qualification hypothesis by Kunter et al., 2013) which reflects the ‘good teachers are made’ argument; (b) the entity view (referred to as the individual aptitude hypothesis by Kunter et al.), which reflects the ‘good teachers are born’ argument; and (c) the dynamic interactionist view, which recognises that good teachers develop through the interaction of relatively stable personal characteristics with environmental factors. For those holding an incremental view, selection is not very important because they believe that key teacher attributes and skills can be developed through effective teacher training and professional development. For those holding an entity view, selection is everything because personal characteristics are largely immutable and hence play a key role in developing an effective teaching force. In this view, teacher training and professional development are less important than selecting teachers with the right set of personal characteristics. Figure 1. Three views on the development of effective teachers. In the dynamic interactionist view, individual differences in teachers’ personal characteristics interact with environmental and learning factors to influence effectiveness in the classroom. Levels of teacher effectiveness increase over time as teachers gain experience (Atteberry et al., 2015) and as they learn new approaches to teaching and interacting with students. The individual ASSESSING ATTRIBUTES OF PROSPECTIVE TEACHERS e 1. Three views on the development of effective teachers. Assessing critical attributes of prospective teachers 9 differences in teacher effectiveness are influenced by within-person cognitive factors (e.g. verbal ability, numerical ability), non-cognitive factors (e.g. personality, motivation, and beliefs about teaching and learning), and by the quality and quantity of learning opportunities provided. We believe that the dynamic interactionist view acknowledges the importance of the cognitive and non-cognitive attributes that candidates bring into ITE programmes and into the profession, but also the interactions of these important personal characteristics with environmental factors. Thus, for those holding an interactionist view, selection is important because within-person characteristics interact with high quality training and development opportunities. The development of teacher effectiveness Teachers differ in many ways, some of which are fleeting (e.g. moods), and others which are more entrenched (e.g. reasoning ability and personality). Although cognitive and non-cognitive attributes are influenced by environmental factors (e.g. learning opportunities and socialisation), there are also individual differences with a biological basis, with high levels of heritability for intelligence, personality, and motivation (Gottschling, Spengler, Spinath & Spinath, 2012; Krapohl et al., 2014). Cognitive factors such as intelligence account for a large part of the heritability of academic achievement, but non-cognitive factors are also heritable. Genetic influences account for more than half of the correlation between academic achievement and non-cognitive factors such as self-efficacy and personality (Krapohl et al., 2014). In the workplace, cognitive abilities have consistently been shown to predict measures of job performance, especially for work roles that are complex and require active information processing and managing simultaneous mental tasks (Murphy, 2012). Research on teacher effectiveness has shown the importance of non-cognitive attributes in the prediction of teacher effectiveness (e.g. Klassen & Tze, 2014), but how these attributes develop and influence teacher effectiveness is less well known. Making a decision about admissions for teacher training programmes represents a prediction about future effectiveness. In his book Talks to Teachers, William James (1899) spoke of the art of teaching, and proposed that good teachers display ‘an additional endowment altogether, a happy tact and ingenuity to tell us what definite things to say and do when the pupil is before us.’ ITE admissions teams’ selection decisions may be guided by government policies (e.g. Department for Education, 2016). Nonetheless, their primary goal is to identify candidates who display the personal attributes and the subject and pedagogical knowledge that are believed to lead to successful outcomes. Teacher effectiveness emphasises the individual teacher as a causal factor in student learning whereas teaching effectiveness refers to the practices associated with successful outcomes, which may be acquired through training and professional development. Research on teacher effectiveness conducted by Kunter and colleagues in their COACTIV programme of research (Kunter et al., 2013) is built on a dynamic interactionist view of teacher effectiveness. In contrast, other models of teacher competence and teacher effectiveness exclusively highlight the role of learning opportunities – teacher education and professional learning – in the development of effectiveness, with little attention paid to within-teacher factors (e.g. Muijs et al., 2014). The COACTIV model of teachers’ professional competence described by Kunter et al. proposes that competence develops over time through the provision of learning opportunities, but that competence is also influenced by critical personal characteristics that are present at entry into teacher training and practice. In this model, education systems and specific school context influence all aspects of teaching and learning through their relationship with learning opportunities, professional competence, and professional practice. In turn, these factors influence student and teacher outcomes. The contextual factors in this model, such as the macrolevel educational system and specific institutional characteristics, provide learning opportunities that interact with existing characteristics of teachers. In terms of the born-or-made question, the 10 Robert M. Klassen & Lisa E. Kim answer, perhaps not surprisingly, is ‘both’. The active engagement in and reflections on learning opportunities are not just dependent on the qualities of the opportunity, but are also influenced by the characteristics of the individual to whom the opportunity is presented. In Figure 2, we present an adaptation of the COACTIV model of the development of teacher effectiveness for prospective teachers. In an ITE context, contextual factors (e.g. overarching national education system, specific characteristics of the ITE programme, and placement school characteristics) play a role in influencing the available learning opportunities that influence teaching competence and teaching behaviours. At the same time, personal characteristics also play a role in shaping teaching competence and teaching behaviours that lead to student and teacher outcomes. The selection process into an ITE programme provides an opportunity to consider personal characteristics that will interact with contextual factors that together influence teacher effectiveness. Once candidates are admitted into an ITE programme, the focus changes from identifying personal characteristics to providing appropriate learning opportunities. The personal characteristics (cognitive and non-cognitive attributes) of pre-service teachers provide a foundation that does not just influence the development of professional competence and professional practice, but also influences how they engage in available learning opportunities. Kunter et al.’s model recognises both entity and incremental views of teacher effectiveness and provides a theoretical foundation to explain individual differences in teacher effectiveness. Figure 2. Model of the development of teacher effectiveness (adapted from Kunter et al., 2013). ASSESSING ATTRIBUTES OF PROSPECTIVE TEACHERS Figure 2. Model of the development of teacher effectiveness (adapted from Kunter et al., 2013). Assessing critical attributes of prospective teachers 11 Non-cognitive attributes associated with teacher effectiveness Research in education and psychology shows that multiple factors are related to teacher effectiveness. These factors include: (a) background factors such as educational record; (b) cognitive attributes such as subject knowledge and expertise, literacy and numeracy skills, pedagogical knowledge, and reasoning abilities; and (c) non-cognitive attributes such as self-efficacy, personality, and beliefs about knowledge. Kunter et al. (2013) showed that non-cognitive attributes (motivation and beliefs about teaching and learning) make an incremental contribution to successful teaching beyond pedagogical content knowledge. The factors related to teacher effectiveness are multifaceted, with non-cognitive attributes making a contribution over-and-above background and cognitive factors. In this section, we review three key non-cognitive attributes that have been shown to be associated with teacher effectiveness. Self-efficacy Motivation is defined as a contextualised and responsive set of wishes, desires, or underlying beliefs that influence people’s movement towards attainment of goals (Pintrich & Schunk, 2002). Self-efficacy is a motivation construct – sometimes referred to as a competence belief – that concerns people’s beliefs about their capabilities to carry out the courses of action required to accomplish particular goals. Motivation beliefs such as self-efficacy show some stability once established (Bandura, 1999), but may fluctuate during ITE (Klassen & Durksen, 2014). Overall, teachers who have higher levels of self-efficacy are more likely to be rated by classroom observers as being instructionally effective (Klassen & Tze, 2014). Teachers’ self-efficacy – a teacher’s belief in the capability to influence student outcomes – has received considerable research attention, with a growing acknowledgement of its influence on student and teacher behaviours. An individual’s self-efficacy beliefs operate as a motivation variable by increasing effort and persistence of the behaviours required for successful goal completion. Research shows that teachers’ self-efficacy is related to job satisfaction (Klassen & Chiu, 2010), level of stress (e.g. Klassen & Chiu, 2011), and quality of relationships with students (Rimm-Kaufman & Hamre, 2010). Klassen and Tze (2014) conducted a metaanalysis of 43 studies representing 9216 participants, investigating the link between teachers’ psychological characteristics and externally measured teaching effectiveness. The relationship between teachers’ self-efficacy and observed teaching performance was significant and of medium magnitude (r=.28, equivalent to Cohen’s d of .58). Growing evidence on the association between teacher efficacy and teaching and learning outcomes is highlighting some new areas of research focus. Although motivation may show some variation over time, the underlying patterns of motivation may be stable. Watt and Richardson (2008) measured the motivation of pre-service teachers during their teacher training programmes in Australia. Using cluster analysis, the researchers found that a sizable proportion of participants with low motivation, the so-called ‘lower engaged desisters,’ showed little change in motivation profiles over the course of the teacher training programme. In other words, the group of pre-service teachers with low motivation were disaffected with teaching at the very beginning of their training and showed little change as their ITE programme progressed, maintaining this disaffection during the first years of their careers. Pre-service teachers’ motivation profiles over the course of an ITE programme in the US showed similar findings (Watt, Richardson & Wilkins, 2014). The implications for ITE programme candidate selection are clear: predictable and stable motivation profiles of pre-service and practicing teachers suggest that selection into teacher training and practice has long-term consequences. 12 Robert M. Klassen & Lisa E. Kim